Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks

Text-to-image (T2I) diffusion models (DMs) have shown promise in generating high-quality images from textual descriptions. The real-world applications of these models require particular attention to their safety and fidelity, but this has not been sufficiently explored. One fundamental question is whether existing T2I DMs are robust against variations over input texts. To answer it, this work provides the first robustness evaluation of T2I DMs against real-world attacks. Unlike prior studies that focus on malicious attacks involving apocryphal alterations to the input texts, we consider an attack space spanned by realistic errors (e.g., typo, glyph, phonetic) that humans can make, to ensure semantic consistency. Given the inherent randomness of the generation process, we develop novel distribution-based attack objectives to mislead T2I DMs. We perform attacks in a black-box manner without any knowledge of the model. Extensive experiments demonstrate the effectiveness of our method for attacking popular T2I DMs and simultaneously reveal their non-trivial robustness issues. Moreover, we provide an in-depth analysis of our method to show that it is not designed to attack the text encoder in T2I DMs solely

Digital financial inclusion and the urban–rural income gap in China: empirical research based on the Theil index

This study examined the effect of digital financial inclusion in reducing the urban–rural income inequality in China. Based on citylevel panel data, the results showed that digital financial inclusion narrowed the urban–rural income gap significantly by boosting economic growth. The results were robust when the core explained variables were replaced. Heterogeneity analysis showed that digital financial inclusion indicates regional differences in narrowing the urban–rural income gap. This study puts forward corresponding countermeasures for the development of digital financial inclusion and adds to the research on this very topical subjec

Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations

This paper reexamines the research on out-of-distribution (OOD) robustness in the field of NLP. We find that the distribution shift settings in previous studies commonly lack adequate challenges, hindering the accurate evaluation of OOD robustness. To address these issues, we propose a benchmark construction protocol that ensures clear differentiation and challenging distribution shifts. Then we introduce BOSS, a Benchmark suite for Out-of-distribution robustneSS evaluation covering 5 tasks and 20 datasets. Based on BOSS, we conduct a series of experiments on pre-trained language models for analysis and evaluation of OOD robustness. First, for vanilla fine-tuning, we examine the relationship between in-distribution (ID) and OOD performance. We identify three typical types that unveil the inner learning mechanism, which could potentially facilitate the forecasting of OOD robustness, correlating with the advancements on ID datasets. Then, we evaluate 5 classic methods on BOSS and find that, despite exhibiting some effectiveness in specific cases, they do not offer significant improvement compared to vanilla fine-tuning. Further, we evaluate 5 LLMs with various adaptation paradigms and find that when sufficient ID data is available, fine-tuning domain-specific models outperform LLMs on ID examples significantly. However, in the case of OOD instances, prioritizing LLMs with in-context learning yields better results. We identify that both fine-tuned small models and LLMs face challenges in effectively addressing downstream tasks. The code is public at \url{https://github.com/lifan-yuan/OOD_NLP}.Comment: Accepted to NeurIPS 2023 Dataset and Benchmark Track. Code is available at \url{https://github.com/lifan-yuan/OOD_NLP

Generative Pretraining in Multimodality

We present Emu, a Transformer-based multimodal foundation model, which can seamlessly generate images and texts in multimodal context. This omnivore model can take in any single-modality or multimodal data input indiscriminately (e.g., interleaved image, text and video) through a one-model-for-all autoregressive training process. First, visual signals are encoded into embeddings, and together with text tokens form an interleaved input sequence. Emu is then end-to-end trained with a unified objective of classifying the next text token or regressing the next visual embedding in the multimodal sequence. This versatile multimodality empowers the exploration of diverse pretraining data sources at scale, such as videos with interleaved frames and text, webpages with interleaved images and text, as well as web-scale image-text pairs and video-text pairs. Emu can serve as a generalist multimodal interface for both image-to-text and text-to-image tasks, and supports in-context image and text generation. Across a broad range of zero-shot/few-shot tasks including image captioning, visual question answering, video question answering and text-to-image generation, Emu demonstrates superb performance compared to state-of-the-art large multimodal models. Extended capabilities such as multimodal assistants via instruction tuning are also demonstrated with impressive performance.Comment: Code and Demo: https://github.com/baaivision/Em

Theory of Subcycle Linear Momentum Transfer in Strong-Field Tunneling Ionization

Interaction of a strong laser pulse with matter transfers not only energy but also linear momentum of the photons. Recent experimental advances have made it possible to detect the small amount of linear momentum delivered to the photoelectrons in strong-field ionization of atoms. We present numerical simulations as well as an analytical description of the subcycle phase (or time) resolved momentum transfer to an atom accessible by an attoclock protocol. We show that the light-field-induced momentum transfer is remarkably sensitive to properties of the ultrashort laser pulse such as its carrier-envelope phase and ellipticity. Moreover, we show that the subcycle-resolved linear momentum transfer can provide novel insights into the interplay between nonadiabatic and nondipole effects in strong-field ionization. This work paves the way towards the investigation of the so-far unexplored time-resolved nondipole nonadiabatic tunneling dynamics. © 2020 authors

From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework

Textual adversarial attacks can discover models' weaknesses by adding semantic-preserved but misleading perturbations to the inputs. The long-lasting adversarial attack-and-defense arms race in Natural Language Processing (NLP) is algorithm-centric, providing valuable techniques for automatic robustness evaluation. However, the existing practice of robustness evaluation may exhibit issues of incomprehensive evaluation, impractical evaluation protocol, and invalid adversarial samples. In this paper, we aim to set up a unified automatic robustness evaluation framework, shifting towards model-centric evaluation to further exploit the advantages of adversarial attacks. To address the above challenges, we first determine robustness evaluation dimensions based on model capabilities and specify the reasonable algorithm to generate adversarial samples for each dimension. Then we establish the evaluation protocol, including evaluation settings and metrics, under realistic demands. Finally, we use the perturbation degree of adversarial samples to control the sample validity. We implement a toolkit RobTest that realizes our automatic robustness evaluation framework. In our experiments, we conduct a robustness evaluation of RoBERTa models to demonstrate the effectiveness of our evaluation framework, and further show the rationality of each component in the framework. The code will be made public at \url{https://github.com/thunlp/RobTest}.Comment: Accepted to Findings of ACL 202

Isolation, purification, and structural elucidation of Stropharia rugosoannulata polysaccharides with hypolipidemic effect

Stropharia rugosoannulata is a widely grown edible mushroom with a high nutritional value. S. rugosoannulata polysaccharides is one of the most important bioactive components of S. rugosoannulata and has a wide range of activities. A S. rugosoannulata polysaccharides, named SRF-3, was derived from the S. rugosoannulata extraction by freeze-thaw combine with hot water extraction method, then prepareed with DEAE-cellulose column and Sephacryl S-200 HR gel column, and its hypolipidemic activity was determined. The structural characteristics of SRF-3 were analyzed by infrared spectral scanning (FT-IR), ultra-high performance liquid chromatography (UHPLC), acid hydrolysis, methylation analysis, nuclear magnetic resonance (NMR), and Gas Chromatography-Mass Spectrometer (GC-MS). SRF-3 is composed of mannose, galactose, methyl galactose and fructose with ratios of 16, 12, 58 and 12, respectively. In addition, the average relative molecular mass of SRF-3 is approximately 24 kDa. The main chain of SRF-3 is mainly composed of repeating α-D-1,6-Galp and α-D-1,6-Me-Galp units, with branches in the O-2 position of Gal. The structure is presumed to be a mannogalactan, with a small amount of t-β-D-Manp present as a side chain. Hypolipidemic activity assay showed that SRF-3 had good antioxidant and hypolipidemic effects in vitro, suggesting that SRF-3 have potential application in reducing liver fat accumulation

Enhanced piezoelectric, electrocaloric and energy storage properties at high temperature in lead-free Bi0.5(Na1-xKx)0.5TiO3ceramics

The piezoelectric, electrocaloric and energy storage properties were systemically investigated in lead-free Bi0.5(Na1-xKx)0.5TiO3 ceramics from room temperature to high temperature region. These ceramics can be poled completely to obtain large piezoelectric coefficient (104–153 pC/N) at low electric field of ~30 kV/cm. The piezoelectric propertyshows good thermal stability due to high depolarization temperature (Td). For BNKT20, a large low electric field-induced strain of 0.36% is obtained at 120 °C under 50 kV/cm, the corresponding normalized strain coefficient is up to 720 pm/V, which is larger than other BNT-based ceramics at high temperature region. The electrocaloric properties of these ceramics are studied via indirect and direct methods. Large EC value (~1.08 K) in BNKT20ceramic is obtained at 50 kV/cm using indirect calculation. Above 100 °C, the dielectricenergy storage density and efficiency of BNKT20 is still up to ~0.85 J/cm3 and 0.75, respectively. The BNKTx ceramics may become promising candidates in the fields of actuators, electrocaloric cooling and energy storage at high temperature region.Validerad;2019;Nivå 2;2019-03-11 (johcin)</p